AI-based framework for automatically extracting high-low features from NDS data to understand driver behavior

Our ability to detect and characterize unsafe driving behaviors in naturalistic driving environments and associate them with road crashes will be a significant step toward developing effective crash countermeasures. Due to some limitations, researchers have not yet fully achieved the stated goal of characterizing unsafe driving behaviors. These limitations include, but are not limited to, the high cost of data collection and the manual processes required to extract information from NDS data. In light of this limitations, the primary objective of this study is to develop an artificial intelligence (AI) framework for automatically extracting high-low features from the NDS dataset to explain driver behavior using a low-cost data collection method. The author proposed three novel objectives for achieving the study's objective in light of the identified research gaps. Initially, the study develops a low-cost data acquisition system for gathering data on naturalistic driving. Second, the study develops a framework that automatically extracts high- to low-level features, such as vehicle density, turning movements, and lane changes, from the data collected by the developed data acquisition system. Thirdly, the study extracted information from the NDS data to gain a better understanding of people's car-following behavior and other driving behaviors in order to develop countermeasures for traffic safety through data collection and analysis. The first objective of this study is to develop a multifunctional smartphone application for collecting NDS data. Three major modules comprised the designed app: a front-end user interface module, a sensor module, and a backend module. The front-end, which is also the application's user interface, was created to provide a streamlined view that exposed the application's key features via a tab bar controller. This allows us to compartmentalize the application's critical components into separate views. The backend module provides computational resources that can be used to accelerate front-end query responses. Google Firebase powered the backend of the developed application. The sensor modules included CoreMotion, CoreLocation, and AVKit. CoreMotion collects motion and environmental data from the onboard hardware of iOS devices, including accelerometers, gyroscopes, pedometers, magnetometers, and barometers. In contrast, CoreLocation determines the altitude, orientation, and geographical location of a device, as well as its position relative to an adjacent iBeacon device. The AVKit finally provides a high-level interface for video content playback. To achieve objective two, we formulated the problem as both a classification and time-series segmentation problem. This is due to the fact that the majority of existing driver maneuver detection methods formulate the problem as a pure classification problem, assuming a discretized input signal with known start and end locations for each event or segment. In practice, however, vehicle telemetry data used for detecting driver maneuvers are continuous; thus, a fully automated driver maneuver detection system should incorporate solutions for both time series segmentation and classification. The five stages of our proposed methodology are as follows: 1) data preprocessing, 2) segmentation of events, 3) machine learning classification, 4) heuristics classification, and 5) frame-by-frame video annotation. The result of the study indicates that the gyroscope reading is an exceptional parameter for extracting driving events, as its accuracy was consistent across all four models developed. The study reveals that the Energy Maximization Algorithm's accuracy ranges from 56.80 percent (left lane change) to 85.20 percent (right lane change) (lane-keeping) All four models developed had comparable accuracies to studies that used similar models. The 1D-CNN model had the highest accuracy (98.99 percent), followed by the LSTM model (97.75 percent), the RF model (97.71 percent), and the SVM model (97.65 percent). To serve as a ground truth, continuous signal data was annotated. In addition, the proposed method outperformed the fixed time window approach. The study analyzed the overall pipeline's accuracy by penalizing the F1 scores of the ML models with the EMA's duration score. The pipeline's accuracy ranged between 56.8 percent and 85.0 percent overall. The ultimate goal of this study was to extract variables from naturalistic driving videos that would facilitate an understanding of driver behavior in a naturalistic driving environment. To achieve this objective, three sub-goals were established. First, we developed a framework for extracting features pertinent to comprehending the behavior of natural-environment drivers. Using the extracted features, we then analyzed the car-following behaviors of various demographic groups. Thirdly, using a machine learning algorithm, we modeled the acceleration of both the ego-vehicle and the leading vehicle. Younger drivers are more likely to be aggressive, according to the findings of this study. In addition, the study revealed that drivers tend to accelerate when the distance between them and the vehicle in front of them is substantial. Lastly, compared to younger drivers, elderly motorists maintain a significantly larger following distance. This study's results have numerous safety implications. First, the analysis of the driving behavior of different demographic groups will enable safety engineers to develop the most effective crash countermeasures by enhancing their understanding of the driving styles of different demographic groups and the causes of collisions. Second, the models developed to predict the acceleration of both the ego-vehicle and the leading vehicle will provide enough information to explain the behavior of the ego-driver.Includes bibliographical references

SigSegment: A Signal-Based Segmentation Algorithm for Identifying Anomalous Driving Behaviours in Naturalistic Driving Videos

In recent years, distracted driving has garnered considerable attention as it continues to pose a significant threat to public safety on the roads. This has increased the need for innovative solutions that can identify and eliminate distracted driving behavior before it results in fatal accidents. In this paper, we propose a Signal-Based anomaly detection algorithm that segments videos into anomalies and non-anomalies using a deep CNN-LSTM classifier to precisely estimate the start and end times of an anomalous driving event. In the phase of anomaly detection and analysis, driver pose background estimation, mask extraction, and signal activity spikes are utilized. A Deep CNN-LSTM classifier was applied to candidate anomalies to detect and classify final anomalies. The proposed method achieved an overlap score of 0.5424 and ranked 9th on the public leader board in the AI City Challenge 2023, according to experimental validation results

AI-Based Framework for Understanding Car Following Behaviors of Drivers in A Naturalistic Driving Environment

The most common type of accident on the road is a rear-end crash. These crashes have a significant negative impact on traffic flow and are frequently fatal. To gain a more practical understanding of these scenarios, it is necessary to accurately model car following behaviors that result in rear-end crashes. Numerous studies have been carried out to model drivers' car-following behaviors; however, the majority of these studies have relied on simulated data, which may not accurately represent real-world incidents. Furthermore, most studies are restricted to modeling the ego vehicle's acceleration, which is insufficient to explain the behavior of the ego vehicle. As a result, the current study attempts to address these issues by developing an artificial intelligence framework for extracting features relevant to understanding driver behavior in a naturalistic environment. Furthermore, the study modeled the acceleration of both the ego vehicle and the leading vehicle using extracted information from NDS videos. According to the study's findings, young people are more likely to be aggressive drivers than elderly people. In addition, when modeling the ego vehicle's acceleration, it was discovered that the relative velocity between the ego vehicle and the leading vehicle was more important than the distance between the two vehicles

Fine-Tuning YOLOv5 with Genetic Algorithm For Helmet Violation Detection

The present study addresses the issue of non-compliance with helmet laws and the potential danger to both motorcycle riders and passengers. Despite the well-established advantages of helmet usage, compliance remains a formidable challenge in many regions of the world, with various factors contributing to the issue. To mitigate this concern, real-time monitoring and enforcement of helmet laws have been advocated as a plausible solution. However, previous attempts at real-time helmet violation detection have been limited by their inability to operate in real-time. To remedy this issue, the current paper proposes a real-time helmet violation detection system utilizing a single-stage object detection model called YOLOv5. The model was trained on the 2023 NVIDIA AI City Challenge Track 5 dataset and employed genetic algorithms in selecting the optimal hyperparameters for training the model. Furthermore, data augmentation techniques such as flip, and rotation were implemented to improve model performance. The efficacy of the model was assessed using mean average precision (mAP). Our developed model achieved an mAP score of 0.5377 on the experimental test data which won 10th place on the public leaderboard. The proposed approach represents a noteworthy breakthrough in the field and holds the potential to significantly improve motorcycle safety

Real-Time Helmet Violation Detection Using YOLOv5 and Ensemble Learning

The proper enforcement of motorcycle helmet regulations is crucial for ensuring the safety of motorbike passengers and riders, as roadway cyclists and passengers are not likely to abide by these regulations if no proper enforcement systems are instituted. This paper presents the development and evaluation of a real-time YOLOv5 Deep Learning (DL) model for detecting riders and passengers on motorbikes, identifying whether the detected person is wearing a helmet. We trained the model on 100 videos recorded at 10 fps, each for 20 seconds. Our study demonstrated the applicability of DL models to accurately detect helmet regulation violators even in challenging lighting and weather conditions. We employed several data augmentation techniques in the study to ensure the training data is diverse enough to help build a robust model. The proposed model was tested on 100 test videos and produced an mAP score of 0.5267, ranking 11th on the AI City Track 5 public leaderboard. The use of deep learning techniques for image classification tasks, such as identifying helmet-wearing riders, has enormous potential for improving road safety. The study shows the potential of deep learning models for application in smart cities and enforcing traffic regulations and can be deployed in real-time for city-wide monitoring

GazeGNN: A Gaze-Guided Graph Neural Network for Disease Classification

The application of eye-tracking techniques in medical image analysis has become increasingly popular in recent years. It collects the visual search patterns of the domain experts, containing much important information about health and disease. Therefore, how to efficiently integrate radiologists' gaze patterns into the diagnostic analysis turns into a critical question. Existing works usually transform gaze information into visual attention maps (VAMs) to supervise the learning process. However, this time-consuming procedure makes it difficult to develop end-to-end algorithms. In this work, we propose a novel gaze-guided graph neural network (GNN), GazeGNN, to perform disease classification from medical scans. In GazeGNN, we create a unified representation graph that models both the image and gaze pattern information. Hence, the eye-gaze information is directly utilized without being converted into VAMs. With this benefit, we develop a real-time, real-world, end-to-end disease classification algorithm for the first time and avoid the noise and time consumption introduced during the VAM preparation. To our best knowledge, GazeGNN is the first work that adopts GNN to integrate image and eye-gaze data. Our experiments on the public chest X-ray dataset show that our proposed method exhibits the best classification performance compared to existing methods

Driver Maneuver Detection and Analysis Using Time Series Segmentation and Classification

The current paper implements a methodology for automatically detecting vehicle maneuvers from vehicle telemetry data under naturalistic driving settings. Previous approaches have treated vehicle maneuver detection as a classification problem, although both time series segmentation and classification are required since input telemetry data are continuous. Our objective is to develop an end-to-end pipeline for the frame-by-frame annotation of naturalistic driving studies videos into various driving events including stop and lane-keeping events, lane changes, left-right turning movements, and horizontal curve maneuvers. To address the time series segmentation problem, the study developed an energy-maximization algorithm (EMA) capable of extracting driving events of varying durations and frequencies from continuous signal data. To reduce overfitting and false alarm rates, heuristic algorithms were used to classify events with highly variable patterns such as stops and lane-keeping. To classify segmented driving events, four machine-learning models were implemented, and their accuracy and transferability were assessed over multiple data sources. The duration of events extracted by EMA was comparable to actual events, with accuracies ranging from 59.30% (left lane change) to 85.60% (lane-keeping). Additionally, the overall accuracy of the 1D-convolutional neural network model was 98.99%, followed by the long-short-term-memory model at 97.75%, then the random forest model at 97.71%, and the support vector machine model at 97.65%. These model accuracies were consistent across different data sources. The study concludes that implementing a segmentation-classification pipeline significantly improves both the accuracy of driver maneuver detection and the transferability of shallow and deep ML models across diverse datasets.This article is published as Aboah, Armstrong, Yaw Adu-Gyamfi, Senem Velipasalar Gursoy, Jennifer Merickel, Matt Rizzo, and Anuj Sharma. "Driver Maneuver Detection and Analysis using Time Series Segmentation and Classification." Journal of Transportation Engineering, Part A: Systems 149, no. 3 (2023): 04022157. DOI: 10.1061/JTEPBS.TEENG-7312. Copyright 2022 American Society of Civil Engineers. Posted with permission

The 1st Data Science for Pavements Challenge

The Data Science for Pavement Challenge (DSPC) seeks to accelerate the research and development of automated vision systems for pavement condition monitoring and evaluation by providing a platform with benchmarked datasets and codes for teams to innovate and develop machine learning algorithms that are practice-ready for use by industry. The first edition of the competition attracted 22 teams from 8 countries. Participants were required to automatically detect and classify different types of pavement distresses present in images captured from multiple sources, and under different conditions. The competition was data-centric: teams were tasked to increase the accuracy of a predefined model architecture by utilizing various data modification methods such as cleaning, labeling and augmentation. A real-time, online evaluation system was developed to rank teams based on the F1 score. Leaderboard results showed the promise and challenges of machine for advancing automation in pavement monitoring and evaluation. This paper summarizes the solutions from the top 5 teams. These teams proposed innovations in the areas of data cleaning, annotation, augmentation, and detection parameter tuning. The F1 score for the top-ranked team was approximately 0.9. The paper concludes with a review of different experiments that worked well for the current challenge and those that did not yield any significant improvement in model accuracy